Papers with WMT14 English
A Mixture of h - 1 Heads is Better than h Heads (2020.acl-main)
Copied to clipboard
| Challenge: | Evidence has shown that multi-head attentive neural architectures are overparameterized. |
| Approach: | They propose a multi-head attentive neural architecture that “reallocates” attention heads to different inputs. |
| Outcome: | The proposed model outperforms baselines on machine translation and language modeling tasks. |